Overcoming Human Weaknesses in Validation of Knowledge-Based Systems

Human experts employed in validation exercises for knowledge-based systems often have limited time and availability. They often have different opinions from each other as well as from themselves over time. We address this situation by introducing the use of validation knowledge used in prior validation exercises for the same knowledge-based system. We present a Validation Knowledge Base (VKB) that is the collective best experience of several human expert validators. We also present the concept of Validation Expert Software Agents (VESA) that represent a particular expert’s knowledge. VESA is a software agent corresponding to a specific human expert validator. It models the validation knowledge and behavior of its human counterpart by analyzing similarities with the responses of other experts. We also describe experiments with a small prototype system to evaluate the usefulness of these concepts.